Exploring Video Structure Beyond the Shots

Yong Rui, Thomas S. Huang and Sharad Mehrotra Beckman Institute for Advanced Science and Technology Dept. of Electrical and Computer Engineering and Dept. of Computer Science University of Illinois at Urbana-Champaign Urbana, IL 61801, USA E-mail:fyrui,[email protected] and [email protected] ABSTRACT While existing shot-based video analysis approaches provide users with better access to the video than the raw data stream does, they are still not su cient for meaningful video browsing and retrieval, since: (1) the shots in a long video are still too many to be presented to the user; (2) shots do not capture the underlying semantic structure of the video, based on which the user may wish to browse/retrieve the video. To explore video structure at a semantic level, this paper presents an e ective approach for video scene structure construction, in which shots are grouped into semantic-related scenes. The output of the proposed algorithm provides a structured video that greatly facilitates user's access. Experiments based on real-world movie videos validate the effectiveness of the proposed approach.